In the classical von Neumann computing architecture, computation and data storage is performed by separate modules, the central processing unit and the random access memory, respectively (cf., e.g., A. N. Burks, H. H. Goldstine, and J. von Neumann. Preliminary discussion of the logical design of an electronic computing instrument. Report 1946, U.S. Army Ordonance Department, 1946) A memory address sent to the random access memory gives access to the data content of one particular storage location. Associative memories are computing architectures in which computation and data storage is not separated (cf., e.g., T. Kohonen. Associative memory: a system theoretic approach. Springer, Berlin, 1977). For example, an associative memory can store a set of associations between pairs of (binary) patterns {(uμ→vμ): μ=1, . . . , M} (see FIG. 1).
FIG. 1 illustrates a memory task: In the storage phase (upper part of FIG. 1), M. associations of memory address patterns uμ and memory content patterns are stored in the associative memory device (AM). In the retrieval phase (lower part of FIG. 1), the AM is addressed with an input query pattern ũ typically resembling one of the previously stored memory address patterns uμ1. The AM returns the retrieval result {circumflex over (v)} that should be similar to the associated memory content pattern vμ1.
Similar as in random access memory, an input query pattern uμ entered in the associative memory can serve as address for accessing the associated pattern vμ. However, the tasks performed by the two types of memory differ fundamentally. Random access is only defined for input query patterns that are valid addresses, that is, for the set of u patterns used during storage. The random access task consists of returning the data record at the addressed location (look-up).
In contrast, associative memories accept arbitrary nut query patterns ũ and the computation of any particular output involves all stored data records rather than a single one. Specifically, the associative memory task consists of comparing an input query pattern ũ with all stored addresses and returning an output pattern equal (or similar) to the pattern vμ associated with the memory address pattern uμ most similar to the input query pattern. Thus, the associative memory task includes the random access task but is not restricted to it. It also includes computations such as pattern completion, de-noising or retrieval using incomplete cues.
Neural associative memories are parallel implementations of associative memory in a network of neurons in which associations are stored in a set of synaptic weights H between neurons typically employing fast Hebbian-type learning methods (cf., j. Hertz, A. Krogh, and R. G. Palmer. introduction to the theory of neural computation. Addison-Wesley, Redwood City, 1991), see FIG. 2. Neural associative networks are closely related to Hebbian cell assemblies (cf., e.g., G. Palm. Neural Assemblies. An Alternative Approach to Artificial Intelligence. Springer, Berlin, 1982) and play an important role in neuroscience as models of neural computation for various brain structures, in particular for neocortex, hippocampus, and cerebellum.
FIG. 2 illustrates a neuronal associative memory (NM considered by this invention. It is a two-layer neuronal network consisting of an address population u (size and a content population v (size All address neuron u; can make synaptic contacts with weight wij onto content neuron vj. When addressing with an input query pattern ũ, a content neuron vj gets active if the dendritic potential xj:=
      ∑          i      =      1        m    ⁢            w      ij        ⁢                  u        ~            i      exceeds the neuron's firing threshold Θj. Memory associations are stored in the synaptic weights and firing thresholds of the network. FIG. 2 shows a hetero-associative memory. For identical u and v the network becomes an auto-associative memory with recurrent synaptic connections.